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Abstract 

The problem of ranking a set of objects given some mea- 
sure of similarity is one of the most basic in machine 
learning. Recently fP] proposed a method based on tech- 
niques in semi-supervised learning utilizing the graph 
Laplacian. In this work we consider a novel application 
of this technique to ranking binary choice data and apply 
it specifically to ranking US Senators by their ideology. 

1 Introduction 

Ranking is one of the most fundamental problems in 
machine learning. Over the years, many algorithms 
have been proposed many of them relying on classical 
techniques such as linear regression. In order to apply 
regression techniques typically we need have access to 
the features of the objects in order to learn the regression 
equation. We can then run the regression equation on the 
test data and output our ranking. 

Recently there has been more interest in the setting 
where we only have access to the similarity among 
objects and we want to output a ranking. The natural 
interpretation of this ranking is that we want similar 
objects to be ranked close to each other and dissimilar 
objects to be ranked far apart from each other This set of 
similarities can be visualized as a graph, where the nodes 
are the objects we want to rank and the weight on the 
edges expresses the similarity of the examples connected 
by that edge. 

The "graph learning" setting has proven very popular 
in machine learning and there have been a slew of 
papers exploring this paradigm (HI B H 01- In a recent 



paper Agarwal IH proposed applying the graph learning 
technique to the problem of ranking. In this paper we 
propose to apply this technique to ranking binary choice 
data, with the specific application of ranking US Senators 
by their ideology. 



2 Background 

2.1 Ranking Legislators 

The problem of ranking politicians by ideology has been 
well studied in the political science literature. Currently 
the most popular such ranking system is Vote View devel- 
oped by political scientist Keith Poole and hosted on his 
web site VoteView.com. Voteview works by projecting 
the set of legislators into a low dimensional Euclidean 
space and then iteratively searching for cutting planes 
that optimally divide the legislators into sets that agree 
with each other 0] . 

In particular we note that this optimization procedure 
is not guaranteed to converge to a global minimum. 
By contrast the method that we present involves only 
basic linear algebra and is guaranteed to give a global 
minimum. In addition it is very easy to implement and 
has a lower running time. 



2.2 The Combinatorial Graph Laplacian 

In our solution we will a matrix known as the combi- 
natorial graph Laplacian (or simply the Laplacian of a 
graph) because of several suggestive analogies with the 
classical Laplacian differential operator widely used in 
physics (among other places). The Laplacian has many 
interesting properties of which we will exploit only a few 
in this work. For more details refer to the thesis of Zhu[6]. 

For an undirected graph G, the Laplacian A is defined 
as the matrix 

A = D-A 

where A is the adjacency matrix of G and D is the diag- 
onal matrix which has the degree of each vertex on the 
diagonal (i.e the degree matrix which is equivalently the 
sum of very row). 

The concept can be straightforwardly generalized to a 
weighted graph by defining 

D-W 

where Wi,j is the weight between nodes i and j and Di^i 
is the sum of row i of matrix W. 



3 Ranking 



Our goal is to develop a method for ranking legislators by 
ideology based on their votes. We can use a simple model 
for this situation: Each senator has a ideology score 
f between —1 and +1 that represents their political 
leanings. For example we can let +1 represent an ex- 
tremely liberal senator and —1 an extremely conservative 
one. Our task is to compute these scores based on their 
votes. 

To derive an algorithm we make a simple and obvious 
assumption: Senators that have a similar voting record 
should have similar ideological scores. That is if two 
senators voting records are very similar we assume their 
political views are also very similar. 

We propose the following as our objective function 
which we will seek to minimize. 

n n 
i j 

where Wij is the similarity between senators i and j 
and n is the number of senators. 

To obtain a ranking we have to find the values of fi 
which minimize this objective function. 

Lemma 1 

n n 
i 3 

where A is the graph Laplacian and / is the vector of 
all the fi's. 

Proof: 

n n 
i j 

n n 

n n 
i 3 
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= E E ^^.i/^ - 2 E E + E E ^^^^fl 
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Now 



E E w^.f! = E(w-.i + + • • • )f! = fDf 

i j i 

Where £) is a diagonal matrix and Di^i is the sum of 
row i of matrix W. 

Similarly 

n n 

EE^^.i/' = /'^^/ 

i j 

Here we use the assumption that matrix W is symmet- 
ric. 

Lastiy 

n n 
i j 

Putting it all together 

n n n n n n 

= E E ^^./i^ - 2 E E + E E 

i j i j i j 

= fDf-2fWf+fDf = 2f{D-W)f = 2fAf 
QED 

In order to get non-trivial results we will have to 
specify the values of at least two of the fi. (Otherwise 
the minimum can be obtained by just setting all the fi to 
zero which is not very useful). It is convenient to set one 
/j to +1 and another /j to — 1, corresponding to the most 
ideologically pure legislators. 

Once we have done this we can spUt up the vector / 
into /l and fu corresponding to the labeled fi and the 
unlabeled fi respectively. Likewise we can rearrange and 
split up A into Auu Aul Alu ^ll as can be seen in 
the following table: 





L 


U 


L 


All 


Alu 


U 


AuL 





Our task now is to minimize 2/^ A/ which is the same 
as minimizing A/ 

Lemma 2 The fu that minimizes f^Af is equal to 

-^UU^ULfL 



Proof: 



4.1.1 Example 



fAf = [flfS] 



All 

AuL 



Alu 
Auu 



II 
fu 



= JIAllIl + flALufu + fuAuLh + fu^uufu 

Differentiating by fjj (since is a constant) and set- 
ting to we get: 

2AuLfL + lAuufu = 
Rearranging we get 



Suppose Senator Rightwinger has voted (NO, YES, AB- 
STAIN) on 3 bills while Senator Leftwinger has voted 
(YES,NO,ABSTAIN) on the same set of bills. Then we 
encode their votes as the vectors (-1,H-1,0) and (h-1,-1,0). 
The Hamming distance of the two vectors is 2 and hence 
we will assign the similarity between the two senators the 
value of : 



1 

2+1 



fu — -AjjIjAjjlJl 



QED 



4 Algorithm 

To summarize, our algorithm is as follows: 

1 . First we compute the similarities between all pairs of 
examples. This will give us a similarity matrix W . 

2. We then compute the graph Laplacian A = D — W 

3. We specify at least two labeled fi to get 



4.2 Selecting the labeled fi 

In essence we have to specify at least two fi to which 
we can confidently assign a label. For our purposes it 
makes the most sense to specify the most extreme exam- 
ples. There are broadly two ways of doing this 

1. Use domain knowledge of the political arena - e.g. 
look at the most extreme legislators in other rank- 
ings by advocacy groups and other parties, look at 
the political ideology of the legislator's home district 
and other external evidence. 

2. Purely internal knowledge from the dataset - For ex- 
ample pick the two legislators with the highest poUt- 
ical difference (lowest similarity). 

In practice we find our algorithm is robust to any rea- 
sonable choice (i.e the rankings will not change drastically 
based on the method). 



4. We compute fu 
ranking. 



^a^jIjAulIl to obtain the final 5 Experimental Results 



Now we just need to specify how we are going to com- 
pute the similarities and how we specify the labeled fi. 

4.1 Computing Similarities 

A legislator can essentially only do 3 things on any par- 
ticular vote, 

1. Vote "YES". 

2. Vote "NO". 

3. Fail to register any vote (e.g. absent, abstaining etc). 

We encode the behavior of each legislator as a vector 
of integers in { — l,0,+l}in the obvious way and define 
the "distance" D between two legislators as the Hamming 
difference of their respective vectors. We then define the 
weight Wij as and thus obtain the weight matrix W. 



We obtained data on roll call votes for the 2007-2008 
session of the US Senate from the web site of Keith T. 
Poole iQl- We removed the votes on which there was 
more than 95% agreement as those were most likely 
ceremonial votes. 



5.1 Using Domain Knowledge 

In this experiment we picked Senator Russell Feingold 
of Wisconsin and Senator Thomas Coburn of Oklahoma 
as our liberal and conservative exemplars. Both of these 
senators have a strong reputation for exemplifying the 
liberal and conservative wings of their respective political 
parties in the Senate. As per our methodology we fixed 
fi — 1 for Senator Feingold and fi = —1 for Senator 
Coburn ran our algorithm and obtained the following 
results: 



Table 1 : Ranking using the Domain Knowledge Method 



Rank 


Name 


Party 


1 


FEINGOLD 


D 


2 


SANDERS 


D 


3 


LEAHY 


D 


4 


DURBIN 


D 


5 


HARKIN 


D 


6 


WYDEN 


D 


7 


BROWN 


D 


8 


WHITEHOUSE 


D 


9 


CARDIN 


D 


10 


MENENDEZ 


D 


11 


KERRY 


D 


12 


CANTWELL 


D 


13 


KOHL 


D 


14 


LAUTENBERG 


D 


15 


KLOBUCHAR 


D 


16 


AKAKA 


D 


17 


MURRAY 


D 


18 


SCHUMER 


D 


19 


REED 


D 


20 


BOXER 


D 


21 


BINGAMAN 


D 


22 


LEVIN 


D 


23 


STABENOW 


D 


24 


REID 


D 


25 


CASEY 


D 


26 


MlKULSKl 


D 


27 


FEINSTEIN 


D 


28 


NELSON 


D 


29 


WEBB 


D 


30 


SALAZAR 


D 


31 


TESTER 


D 


32 


INOUYE 


D 


33 


ROCKEFELLER 


D 


34 


KENNEDY 


D 


35 


CONRAD 


D 


36 


DODD 


D 


37 


DORGAN 


D 


38 


CARPER 


D 


39 


BAUCUS 


D 


40 


BIDEN 


D 


41 


MCCASKILL 


D 


42 


LINCOLN 


D 


43 


BYRD 


D 


A 1 
44 


ULliN i UiN 


p\ 
JJ 


45 


LIEBERMAN 


D 


46 


PRYOR 


D 


47 


BAYH 


D 


48 


OBAMA 


D 


49 


LANDRIEU 


D 


50 


NELSON 


D 


51 


JOHNSON 


D 



Table 2: Ranking Using the Domain Knowledge Method 



Rank 


Name 


Party 


52 


SNOWE 

k^X ^ V-^ T T -1—/ 


R 


53 


COLLINS 


R 


54 


SPECTER 


R 


55 


SMITH 


R 


56 


COLEMAN 


R 


57 


WICKER 


R 


58 


VOINOVICH 


R 


59 


THOMAS 


R 


60 


STEVENS 


R 


61 


LUGAR 


R 


62 


MURKOWSKI 


R 


63 


DOMENICI 


R 


64 


WARNER 


R 


65 


HAGEL 


R 


66 


MCCAIN 


R 


67 


LOTT 


R 


68 


COCHRAN 


R 


69 


BENNETT 


R 


70 


HATCH 


R 


71 


BOND 


R 


72 


MARTINEZ 


R 


73 


ROBERTS 


R 


74 


ALEXANDER 


R 


75 


GRASSLEY 


R 


76 


HUTCHISON 


R 


77 


DOLE 


R 


78 


SUNUNU 


R 


79 


BROWNBACK 


R 


80 


CORKER 


R 


81 


CRAIG 


R 


82 


SHELBY 


R 


83 


CRAPO 


R 


84 


BARASSO 


R 


85 


GREGG 


R 


86 


MCCONNELL 


R 


87 


THUNE 


R 


88 


ISAKSON 


R 


89 


CHAMBLISS 


R 


90 


GRAHAM 


R 


91 


VITTER 


R 


92 


CORNYN 


R 


93 


SESSIONS 


R 


94 


BUNNING 


R 


yj 


1/ VI 
JV 1 L 


n 
K 


96 


ENZl 


R 


97 


BURR 


R 


98 


ALLARD 


R 


99 


ENSIGN 


R 


100 


INHOFE 


R 


101 


DEMINT 


R 


102 


COBURN 


R 



Table 3: Ranking Using the Internal Knowledge Method 



Rank 


Name 


Party 


1 


MENENDEZ 


D 


2 


LAUTENBERG 


D 


3 


SCHUMER 


D 


4 


DURE IN 


D 


5 


CANTWELL 


D 


6 


MURRAY 


D 


7 


CARDIN 


D 


8 


BROWN 


D 


9 


WHITEIIOUSE 


D 


10 


BOXER 


D 


11 


REED 


D 


12 


KERRY 


D 


13 


HARKIN 


D 


14 


SANDERS 


D 


15 


LEAHY 


D 


16 


AKAKA 


D 


17 


BINGAMAN 


D 


18 


LEVIN 


D 


19 


STABENOW 


D 


20 


WYDEN 


D 


21 


KOHL 


D 


22 


FEINSTEIN 


D 


23 


KLOBUCHAR 


D 


24 


MlKULSKl 


D 


25 


REID 


D 


26 


CASEY 


D 


27 


NELSON 


D 


28 


FEINGOLD 


D 


29 


SALAZAR 


D 


30 


WEBB 


D 


31 


KENNEDY 


D 


32 


ROCKEFELLER 


D 


33 


INOUYE 


D 


34 


CONRAD 


D 


35 


CARPER 


D 


36 


DORGAN 


D 


37 


TESTER 


D 


38 


BAUCUS 


D 


39 


LINCOLN 


D 


40 


DODD 


D 


41 


BIDEN 


D 


42 


BYRD 


D 


43 


LIEBERMAN 


D 


A 1 
44 


ULliN i UiN 


p\ 
JJ 


45 


PRYOR 


D 


46 


MCCASKILL 


D 


47 


LANDRIEU 


D 


48 


BAYH 


D 


49 


OBAMA 


D 


50 


NELSON 


D 


51 


JOHNSON 


D 



Table 4: Ranking Using the Internal Knowledge Method 



Rank 


Name 


Party 


52 


SNOWE 

k^X ^ V-^ T T -1—/ 


R 


53 


COLLINS 


R 


54 


SPECTER 


R 


55 


SMITH 


R 


56 


COLEMAN 


R 


57 


WICKER 


R 


58 


THOMAS 


R 


59 


VOINOVICH 


R 


60 


STEVENS 


R 


61 


MURKOWSKl 


R 


62 


LUGAR 


R 


63 


MCCAIN 


R 


64 


DOMENICI 


R 


65 


WARNER 


R 


66 


HAGEL 


R 


67 


LOTT 


R 


68 


COCHRAN 


R 


69 


HATCH 


R 


70 


ROBERT 


R 


71 


BENNETT 


R 


72 


MARTINEZ 


R 


73 


ALEXANDER 


R 


74 


BOND 


R 


75 


GRASSLEY 


R 


76 


BROWNBACK 


R 


77 


HUTCHISON 


R 


78 


CORKER 


R 


79 


DOLE 


R 


80 


SUNUNU 


R 


81 


BARASSO 


R 


82 


CRAIG 


R 


83 


SHELBY 


R 


84 


CRAPO 


R 


85 


THUNE 


R 


86 


ISAKSON 


R 


87 


GREGG 


R 


88 


MCCONNELL 


R 


89 


CHAMBLISS 


R 


90 


GRAHAM 


R 


91 


SESSIONS 


R 


92 


VITTER 


R 


93 


CORNYN 


R 


94 


BUNNING 


R 




niN/.! 


n 
K 


96 


BURR 


R 


97 


ALLARD 


R 


98 


KYL 


R 


99 


ENSIGN 


R 


100 


INHOFE 


R 


101 


COBURN 


R 


102 


DEMINT 


R 



5.2 Using Internal Knowledge 



References 



We did another experiment where we picked as our two 
exemplars the senators who were the least similar in 
terms of their voting records. In this case our algorithm 
ended up picking Senator Robert Menendez of New 
Jersey ande Senator Jim DeMint of South Carolina. 
This is interesting because while Senator DeMint has 
a reputation in the Senate as a staunch conservative, 
Senator Menendez does not have as high a public profile. 
This analysis suggests his voting record may be more 
partisian than his low profile reputation suggests. As per 
our methodology we fixed fi = 1 for Senator Menendez 
and fi = —1 for Senator DeMint and ran our algorithm 
and obtained the results shown on the previous page. 



6 Discussion 
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First we note that the rankings produced are very reason- 
able and correlate well with rankings produced by interest 
groups and political commentators. The advantage of an 
data driven method of course is that it does not require 
human expertise. Secondly the "domain knowledge" and 
"internal knowledge" methods produce very similar re- 
sults. This suggests a certain degree of robustness. The 
"internal evidence" method appears preferable as it does 
not require any choice of parameters. 



6.1 Conclusions 

We have presented a fast method for ranking legislators 
based on their votes. The method gives reasonable results, 
is easy to implement and apparently more straightforward 
than competing methods such as that of Poole. In addition 
the similarity matrix is an intuitive concept and suggests 
some applications in the area of visuaUzing the legislature. 



6.2 Future Work 

One interesting idea is to explore the idea of using differ- 
ent similarity functions. In this work we used the plain 
vanilla Hamming distance, about the simplest things that 
we could use. It is possible that a more sophisticated do- 
main specific similarity function might produce qualita- 
tively different result (e.g. diffusion kernel or Rank Sim- 
ilarity). Another idea is to further explore any significant 
quaUtative differences with other ranking algorithms to 
establish the respective advantages and disadvantages of 
the various methods. 



